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LONG-TERM  GOALS 

The  long-term  goal  of  this  project  is  to  develop  robust  global  and  mesoscale  ensemble  analysis  and 
forecast  systems  that  are  able  to  provide  probabilistic  forecast  guidance  in  an  operational  environment. 
Although  most  operational  centers  rely  solely  on  single  deterministic  forecasts  from  numerical  weather 
prediction  models,  recent  advances  in  probabilistic  prediction,  or  ensemble  forecasting,  have  made  this 
a  very  powerful  tool  for  providing  more  complete  input  on  environmental  conditions,  including  a 
measure  of  the  forecast  confidence.  Since  the  atmosphere  is  inherently  chaotic,  the  model  predictions 
of  most,  if  not  all  phenomena  will  always  be  sensitive  to  the  model  initial  conditions  and  physical 
parameterizations.  Mesoscale  models  are  additionally  constrained  by  potential  errors  inherent  in  the 
specification  of  the  lateral  boundary  conditions  (LBC).  Due  to  these  inevitable  uncertainties,  forecasts 
are  most  appropriately  viewed  in  a  probabilistic  sense;  that  is,  forecasts  should  provide  probabilities  of 
the  occurrence  of  specific  high-impact  events,  as  well  as  estimates  of  forecast  skill.  Recent  research  at 
NRL  and  other  research  centers  has  demonstrated  that  probabilistic  information  can  be  obtained  from 
ensemble  of  forecasts  created  from  equally  plausible  initial  states  and  equally-likely  model 
formulations.  However,  such  a  probabilistic  prediction  of  Navy  relevant  high-impact  weather  will 
significantly  benefit  sea  strike  and  sea  shield  functions  only  if  there  are  clearly  identified  user-relevant 
norms.  For  example,  METOC  impacts  on  operations  are  typically  listed  in  a  “stoplight”  format,  often 
with  multiple  weather  criteria  that  must  be  satisfied  (e.g.,  for  the  Predator,  precipitation,  winds,  ceiling, 
visibility,  cloud  cover,  icing,  turbulence,  and  temperature).  Probabilistic  forecasts  allow  for  a  different 
probability  threshold  to  be  set  for  each  criterion  individually  (e.g.  a  90%  probability  for  winds,  but 
only  a  70%  probability  for  visibility),  reflecting  the  relevant  importance  of  each  weather  factor. 
Ensembles  also  allow  for  the  covariance  between  relevant  weather  variables  to  be  taken  into  account. 

In  a  particular  instance,  wind  and  visibility  may  have  a  wide  range  of  values  they  might  take  on  (broad 
probability  distribution  functions),  but  information  in  the  covariance  could  tell  you  that  if  the  wind 
speed  is  high,  then  visibility  will  also  be  high. 

A  prototype  global  Ensemble  Transform  (ET)  ensemble  system  has  been  developed  as  part  of  research 
performed  in  recent  NRL  base  projects  on  global  modeling  and  predictability  (Bishop  and  Toth  1999, 
McLay  et  al.  2007).  In  addition,  a  prototype  ET  mesoscale  ensemble  system,  driven  by  this  global 
ensemble,  has  been  developed  within  the  NRL  6.2  project  "Probabilistic  Prediction  of  High-Impact 
Weather"  that  was  approved  for  FY06-FY09.  The  work  in  this  NRL  base  project  on  mesoscale 
ensembles  has  been  incorporated  into  this  RTP  project,  and  the  funding  for  it,  and  the  additional  ONR 
and  PMW-180  funding  approved  for  this  RTP,  will  allow  for  both  the  global  and  mesoscale  ensemble 
systems  to  be  more  fully  developed,  tested,  validated,  and  transitioned  to  operations  during  the  time 
period  of  FY07-FY09. 
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OBJECTIVES 


The  objective  of  this  project  is  to  incrementally  develop  and  test  an  integrated  multi-scale  ensemble 
modeling  framework  and  forecasting  system,  and  validate  and  transition  it  to  operations  at  the  Fleet 
Numerical  Meteorology  and  Oceanography  Center  (FNMOC).  This  system  will  be  a  state  of  the  art 
global  and  mesoscale  ensemble  forecasting  system  using  the  Navy  Operational  Global  Atmospheric 
Prediction  System  (NOGAPS)  and  the  Coupled/Ocean  Atmosphere  Mesoscale  Prediction  System 
(CO  AMPS®1).  It  will  incorporate  consideration  of  uncertainty  in  the  initial  state  (and  lateral 
boundaries  where  appropriate)  along  with  consideration  of  uncertainty  in  the  model  formulation.  It  will 
be  a  user-friendly,  easily  re-locatable  system  flexible  enough  to  incorporate  new  advances  in  data 
assimilation  and  post-processing  schemes.  The  system  will  provide  high  fidelity,  dynamically 
consistent  probabilistic  forecasts,  and  estimates  of  forecast  uncertainty  for  the  battlespace 
environment.  These  global  and  mesoscale  ensemble  forecasts  will  lead  to  superior  probabilistic 
weather  forecasts  that  will  directly  improve  tactical  decision  aids  (TDA)  for  support  of  FORCENET 
and  the  Sea  Power  2 1  pillars  (Sea  Basing,  Sea  Shield,  and  Sea  Strike),  and  for  applications  to 
homeland  security  and  the  global  war  on  terrorism  (GWOT). 

APPROACH 

Our  approach  is  to  take  existing  prototype  global  and  mesoscale  ensemble  forecast  systems,  built 
through  current  and  previous  NRL  in-house  funding,  and  incrementally  develop  these  individual 
systems  into  an  operationally  feasible,  next-generation,  multi-scale  ensemble  prediction  system.  This 
system  will  consist  of  (a)  the  representation  of  initial  and  model  uncertainty  in  NOGAPS,  (b)  the 
representation  of  initial,  lateral  boundary  condition  (LBC),  and  model  uncertainty  in  COAMPS,  (c)  an 
integrated,  robust  global-mesoscale  ensemble  infrastructure  (to  ensure  consistent  perturbations 
between  the  global  and  mesoscale  systems,  result  in  improvements  to  the  global  system  having  an 
immediate  impact  on  the  mesoscale  system,  and  promote  ease  of  use),  (d)  an  ensemble  verification 
system,  including  scorecard,  and  (e)  a  demonstration  of  performance  capabilities  in  an  operational 
setting.  The  system  will  be  thoroughly  tested  using  scientific  studies  and  norms  and  comprehensive 
ensemble  forecast  experiments. 

WORK  COMPLETED 

For  the  development  of  the  global  ensemble  system,  a  comparison  between  the  new  ET  (Bishop  and 
Toth  1999)  and  the  current  operational  Bred  Vector  (BV,  Toth  and  Kalnay  1993)  scheme  has  been 
completed  for  two  summer  and  two  winter  months  with  a  24-h  cycling  period.  In  addition,  experiments 
for  June- July  2005  were  performed  to  investigate  i)  the  impact  of  a  6-h  cycling  period,  ii)  the  impact  of 
rescaling  the  initial  perturbations  to  smaller  size,  and  iii)  the  use  of  a  64-member  ensemble  rather  than 
a  32  member  ensemble.  Research  was  also  completed  investigating  the  performance  of  ensembles 
with  and  without  the  addition  of  stochastic  perturbations  to  the  tendencies  produced  by  the  cumulus 
convection  parameterization  (Teixeira  and  Reynolds,  2007).  Finally,  research  was  performed  using  an 
archive  of  ensemble  perturbations  to  stochastically  enhance  the  ET  scheme  in  regions  where  the  initial 
ET  perturbations  were  deficient  in  matching  analysis  error  variance  estimates. 
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For  the  development  of  the  mesoscale  ensemble  system,  the  new  ET  method  for  generating  initial 
perturbations  has  also  been  implemented  within  COAMPS.  Perturbed  lateral  boundary  conditions  to 
drive  the  ensemble  were  obtained  from  the  global  NOGAPS  ET  ensemble  (McLay  et  al.,  2007a,  b). 
The  29-member  COAMPS  ensemble  system  has  been  tested  by  running  it  over  a  three  week  period  for 
summer  2005.  To  quantitatively  evaluate  the  perfonnance  of  the  mesoscale  ensemble,  a  large  suite  of 
diagnostics  was  developed  and  employed.  A  method  for  predicting  forecast  error  covariance  due  to  the 
misspecification  of  parameters  governing  the  effect  of  sub-grid  scale  processed  on  grid  box  means  was 
also  developed.  The  performance  of  this  “multi-parameter”  ET  ensemble  was  tested  over  the  same 
three  week  period  as  the  ET  ensemble  without  perturbed  parameters.  The  ability  of  the  multi¬ 
parameter  ET  ensemble  to  work  with  high  resolution  nests  was  tested  by  applying  it  to  the  problem  of 
simulating  urban  meteorology  over  Tokyo  at  1.67  km  resolution  using  1 1  members.  Work  is 
underway  to  create  sea  surface  temperature  (SST)  perturbations  and  investigate  their  impact  on 
ensemble  forecasts. 

RESULTS 

For  the  global  ensemble  system,  it  has  been  shown  that  the  new  ET  scheme  is  superior  to  the 
operational  BV  scheme  under  a  variety  of  metrics,  including  the  RMS  error  of  the  ensemble  mean,  the 
relationship  between  ensemble  variance  and  forecast  error  variance,  probabilistic  (brier)  skill  scores, 
and  the  effective  number  of  directions  spanned  by  the  ensemble  subspace  (Fig.  1),  for  a  number  of 
different  fields  of  interest  to  the  Navy,  including  low-level  and  upper-level  winds  (McLay  et  al. 

2007a). 


24  h 


1 20h 


Fig.  1.  The  normalized  amplitude  of  the  eigenvalues  of  the  global  ensemble  perturbation  covariance 
matrices  after  24  hours  (top)  and  120  hours  (bottom),  for  the  new  ET  scheme  (solid  circles)  and  the 
operational  bred  vector  scheme  (open  squares).  Values  closer  to  1  indicate  less  redundancy  between 
ensemble  members.  The  global  ET  ensemble  exhibits  significantly  less  redundancy  among 
ensemble  members  than  the  operational  scheme  at  both  longer  and  shorter  forecast  periods. 
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In  addition,  it  was  shown  that  the  short  (6-h)  cycling  period  produced  ensembles  that  better  matched 
the  analysis  error  variance  estimates  produced  by  the  NRL  Atmospheric  Variational  Data  Assimilation 
System  (NAVDAS)  than  the  24-h  cycling  period.  The  scaling  of  the  initial  perturbations,  in  general, 
resulted  in  a  degradation  of  the  ensemble  performance.  It  was  also  shown  that  the  improvement  in  skill 
using  64  members  instead  of  32  members  was  marginal.  On  the  other  hand,  the  addition  of  stochastic 
cumulus  perturbations  to  the  ET  ensemble  resulted  in  significant  improvements  under  a  variety  of 
metrics  in  the  tropics  (Fig.  2),  with  little  impact  in  the  extra- tropics.  In  addition,  the  ET  forecast 
ensemble  was  post-processed  with  balanced,  correlated,  growth-conditioned  stochastic  perturbations 
generated  from  a  historical  archive  of  forecast  perturbations.  With  the  post-processing,  the  ET  forecast 
ensemble  showed  a  significantly  improved  match  between  the  ensemble  variance  and  the  analysis  error 
variance  estimates  from  NAVDAS  (McLay  et  al.  2007b). 


TROPICS:  72-h  Forecast:  IQ-m  Wind 


Fig.  2.  The  relationship  between  ensemble  variance,  or  “ ensemble  spread”,  and  forecast 
error  variance,  or  “ forecast  error”,  for  the  10-m  wind  speed  for  the  72-h  forecasts  in  the  tropics 
for  the  operational  global  ensemble  (black  solid  curve  with  cross  hatches),  new  ET  scheme 
(red  solid  curve  with  diamonds)  and  new  ET  scheme  including  stochastic  convection  (blue 
dashed  curve  with  diamonds).  Ideally,  the  curves  have  a  slope  of  1  and  span  a  large 
range  of  forecast  error  values.  The  new  global  ET  scheme  is  an  improvement  on  the  operational 
BV  scheme.  The  addition  of  stochastic  convection  results  in  even  further  improvements. 
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For  the  mesoscale  ensemble  system,  the  29-member  COAMPS  48-h  forecasts  for  the  45-km  CONUS 
domain  for  three  weeks  (25  June  to  17  July  2005)  show  that  the  RMS  error  of  the  ET  ensemble  mean  is 
less  than  that  of  the  unperturbed  control  forecast  (Fig.  3). 


Fig.  3.  COAMPS  48-h  ensemble  forecasts  for  the  45-km  CONUS  domain  from  25 
June  to  17  July  2005  of  the  vertical  profile  of  RMS  u-wind  (m  s'1)  of  the  ensemble 
mean  (magenta)  and  the  control  member  without  ET  (black). 


Additional  experiments  were  conducted  in  which  perturbations  to  the  COAMPS  planetary  boundary 
layer  (PBL),  surface  layer,  microphysical,  and  cumulus  physical  parameterizations  were  included 
along  with  the  ET.  For  the  28  perturbation  members  of  the  ensemble,  the  perturbations  included:  3 
PBL,  2  surface  flux,  2  microphysics,  13  cumulus,  7  PBL/surface  flux/cumulus  combination,  and  1 
PBL/surface  flux  combination.  While  the  ensemble  variance  was  found  to  be  a  good  predictor  of 
forecast  error  variance  without  perturbed  parameters,  the  skill  of  the  ensemble  as  represented  by  the 
spread-skill  relationship  was  further  improved  with  perturbed  parameters  (Fig.  4).  In  addition,  all 
members  were  found  to  be  numerically  stable  with  no  failure  rates  even  though  the  period  considered 
included  two  tropical  cyclones. 
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Fig.  4.  The  COAMPS  48-h  ensemble  forecasts  for  the  45-km  CONUS  domain  from  25  June  to  17 
July  2005  of  the  850-hPa  u-wind  (m  s'1)  ensemble  variance  versus  innovation  variance  for 
COAMPS  forecasts  without  perturbed  physics  (black)  and  with  perturbed  physics  (blue). 


For  the  1.67-km  Tokyo  simulations,  the  COAMPS  urban  canopy  parameterization  was  used  (Holt  and 
Pullen  2007)  with  perturbations  constructed  based  upon  urban  morphologies  to  include  modifications 
to  the  mean  building  heights,  roof  fractions,  anthropogenic  heating,  and  building  radiation  extinction. 
Results  show  that  the  perturbed  parameters  actually  improved  the  ability  of  the  ensemble  to  distinguish 
large  forecast  error  variance  from  small  forecast  error  variance  at  high  resolution  (Fig.  5). 
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Fig.  5.  Ensemble  spread-forecast  error  relationship,  similar  to  Fig.  4,  but  for  11 -member  CO  AMPS 
ensemble  for  2-m  air  temperature  for  6-h  forecasts  on  nest  4  (1.67-km  horizontal  resolution)  for  the 
Tokyo  metropolitan  region  for  25-30  June  2005.  The  gray  line  represents  the  basis  COAMPS  run 
without  perturbed  physics  and  the  blue  line  represents  the  COAMPS  run  with  perturbed  physics. 


IMPACT/APPLICATIONS 

Since  results  indicate  improved  performance  of  the  global  ET  system  over  the  BV  under  a  variety  of 
metrics,  this  new  scheme,  once  transitioned,  should  result  in  more  skillful  global  ensemble  forecasts 
for  the  Navy.  This  should  have  a  direct  impact  on  current  and  planned  ensemble-based  products,  such 
as  the  probability  of  gale-fore  winds,  as  well  as  other  products  that  are  currently  forced  by  the  global 
ensemble,  or  will  be  in  the  future,  including  the  mesoscale  atmospheric  ensembles,  and  surface  ocean 
wave  ensembles.  In  the  future,  it  is  hoped  that  these  more  skillful  probabilistic  tools  will  lead  to  more 
applications  of  probabilistic  forecasts  in  the  DoD,  including  ship  and  air  routing  applications. 

TRANSITIONS 

Based  on  the  results  shown  above,  new  ET  code  has  been  transitioned  through  6.4  to  FNMOC  (3ld 
Quarter  07)  to  run  the  ET  with  a  6-h  cycling  period  and  no  initial  scaling.  The  ET  forecast  ensemble 
will  be  run  with  96  members  for  the  6-h  cycle,  and  a  16-member  subset  of  these  96  members  will  be 
run  to  the  240-h  lead  time.  One  of  the  thoughts  behind  transitioning  a  96-member  ensemble  was  to  get 
FNMOC  used  to  running  a  large  ensemble  and  pave  the  way  for  ensemble  data  assimilation.  Another 
was  that  a  tradeoff  had  to  be  made  between  increased  ensemble  size  over  the  6-h  cycling  interval  and 
increased  ensemble  size  over  the  whole  240-h  forecast  interval.  It  was  indicated  that  FNMOC  would 
only  approve  a  very  small  increase  in  ensemble  size  over  the  whole  240-h  forecast  interval  (say  2-4 
members).  So,  we  traded  the  expense  of  2  additional  240-h  integrations  for  80  additional  members 
over  the  6-h  cycling  interval.  It  is  anticipated  that  the  stochastic  physics  research  will  be  transitioned 
through  6.4  to  operations  some  time  during  FY08. 
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In  addition,  as  part  of  the  Navy/Air  Force  Joint  Mesoscale  Ensemble  (JME)  ensemble  forecasting 
project  we  anticipate  helping  FNMOC  set  up  a  real  time  ensemble  forecasting  system  over  the  Korean 
peninsula  during  2008. 

RELATED  PROJECTS 

The  global  ET  and  Stochastic  Physics  research  began  in  preceding  years  under  the  NRL  base-funded 
project  6.1 -Quantifying  Limits  of  Atmospheric  Predictability  and  the  NOAA  funded  project  6.2 
Stochastic  Physics.  In  addition,  it  is  anticipated  that  research  under  the  NRL  base-funded  project  6.2- 
Model  Discovery,  which  will  use  ensembles  to  understand  model  behavior  within  parameter  space, 
will  eventually  lead  to  methods  to  perturb  parameters  in  ensembles  and  be  transitioned  to  operations 
through  this  work  unit  in  FY09. 

For  the  mesoscale  ensemble  development  related  projects  include  ONR  funded  6.2-Probabilistic 
Forecasting  of  High  Impact  Weather,  6.2-Hidden  Volatility  in  Environmental  State  Estimation,  and 
6.1 -Quantifying  Limits  of  Atmospheric  Predictability.  The  research  is  also  related  to  the  NOAA 
funded  projects  6.2  Stochastic  Physics  and  6.2  Huge  Ensemble  State  Estimation. 
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